Lptest Acknowledgements We Thank 4 Experimental Results Group C: Constrained Error Minimiza- Tion Procedures Group A: Error Correction Procedures the Learning Behavior of Single Neuron Classiiers on Linearly Separable or Nonseparable Input

HK FCR PER sep non inc sep non inc sep non inc separable 452 272 31 149 449 0 3 428 0 24 nonseparable 392 0 1 391 0 4 388 0 0 392 Table 1: Conclusion on linear separability by each algorithm (entries are number of problems; sep: separable, non: nonseparable, inc: inconclusive). results that there is no known proof of convergence for HK and FCR in a predictable number of steps, and that linear programming has known, predictable time bounds for convergence on both linearly separable and nonseparable cases. The implementational diiculties bring doubts on the practicality of these adaptive algorithms. The only reservation we have about this conclusion is that for linear programming we used the MINOS solver 13] through the AMPL interface 4], which was highly optimized commercial code, whereas the adap-tive procedures were run under the simplest implementation by ourselves in C, so aaordable (elapsed) time may not mean the same thing for the two groups. Also, we have not tested the dependence of the run time on the order in which the input vectors were presented, and we have not investigated dual problems that can be formulated for a given problem and solved by any of these procedures 15]. Nevertheless, we advocate that linear programming methods deserve more serious attention in classiication studies. Without more sophisticated derivatives such as simultaneous primal-dual algorithms, the only apparent advantages of the adaptive procedures such as HK and FCR rules seem to be that (1) they can be implemented on very simple machines; and (2) their adaptive nature permits easier inclusion of new input that may become available during the training process, and thus they are better suited for on-line learning. and George Nagy for helpful discussions and suggestions of references. Mitra Basu thanks Bell Labs for the summer support in 1998. Repository of machine learning databases for large problems. So the adaptive algorithm (7) was used instead. With LP, we used the simple formulation (8) to test for linear separability (referred to as LPtest), and also Smith's formulation (9) to derive a minimum error separating hyperplane (referred to as LPme). We included the perceptron training rule too for comparison purpose (PER), although it is understood that it does not converge for nonseparable input. The problems were discrimination between all pairs of classes in 14 data sets from the UC-Irvine Machine Learning Depository 2]. The data sets were …